October 24, 2023
By Bob O'Donnell
After many years of monotony and relative stasis in the PC industry, things are really starting to change. And man, it’s getting exciting again!
Earlier this year, AMD launched the Ryzen 7040, the first PC-focused SOC (system on chip) with a built-in NPU (Neural Processing Unit)—otherwise known as an AI accelerator. Just a few weeks ago, Intel underlined this progress with the debut of its Core Ultra chip (codenamed “Meteor Lake”) featuring its own AI accelerator. Now, Qualcomm is putting an exclamation point on the AI PC trend with the launch of its Snapdragon X Elite SOC for PCs.
Not only does the Snapdragon X Elite feature a powerful NPU—in fact, it has about 4X the raw performance of AMD and Intel NPUs when measured on a TOPs (Tera Operations per Second) basis—it also marks the long-awaited debut of the Oryon CPU. Based on technology that Qualcomm acquired several years back from a company called Nuvia, Oryon is an Arm-based CPU that offers surprisingly strong capabilities. In its first implementation of the technology, Qualcomm is combining 12 CPU cores running at 3.8 GHz (two of which can boost up to 4.3 GHz) and building the chip on a 4 nm process. Interestingly, while Qualcomm is initially using Oryon for PCs, it’s expected to become the CPU core in future generation mobile, automotive, XR and other application chips as well.
On a PC, the performance looks to be pretty impressive. Putting it more practically, the benchmarks that Qualcomm displayed at its Snapdragon Summit event showed that Oryon-equipped Snapdragon X Elite chips can beat the speed of Apple’s M2 Max chip in single-threaded performance or match its multi-threaded performance with 30% less power. They also beat the performance of Intel’s i9-13980Hx in single-threaded performance or matched its multi-threaded performance with 70% less power. For multi-threaded performance, the numbers were equally impressive, with Qualcomm saying it was 50% better than Apple’s M2 and up to 2x faster than some Intel CPUs.
Of course, as with most everything in the computing world, your mileage may vary (YMMV) depending on the specific applications you need to run. Still, there’s no doubt that it’s an impressive leap forward in performance and brings a new sense of relevancy to Arm-based PCs…or it will, that is, when systems sporting the Snapdragon X Elite become available sometime around the middle of next year.
In addition to impressive new capabilities in CPU performance, the AI acceleration features of Snapdragon X Elite are generating a lot of buzz across the PC industry, especially because of the growing interest in generative AI (GenAI). While most of the GenAI focus until now has been on cloud-based applications and services, it’s becoming increasingly clear that the possibility of running foundation model-powered LLMs (Large Language Models) and other GenAI applications directly on PCs is coming much sooner than expected—especially with products like Snapdragon X Elite. In fact, Qualcomm discussed the fact that it will be able to run GenAI models with up to 13 billion parameters directly on PCs with Snapdragon X Elite. This opens up the potential for a number of very powerful applications, including things like Meta’s Llama 2, Stable Diffusion and much more, running locally.
Running LLMs and other GenAI models directly on the device offers a number of security and privacy enhancements versus running them in the cloud, particularly for digital assistant and other types of future applications that can leverage the work you do on your PC. Plus, for certain applications that are trained specifically for PCs, many AI experts have said you’ll even be able to get better performance locally than from a cloud-based version. Of course, in some situations, there’s little doubt that the full power of cloud computing will be a better choice, but it’s definitely not always going to be the case—and that’s a big step forward in credibility for AI PCs.
Another aspect of the Snapdragon X Elite specs that should help its AI performance is support for LPDDR5x memory and impressive memory transfer speeds of 136 GB/s. Many GenAI models are very memory dependent, so quick access to large amounts of memory will increasingly become an important spec to look for in AI PCs. The new chip also has an improved version of Qualcomm’s Adreno GPU technology, which the company claims is up to 2x faster than the competitive integrated GPU offerings from Intel and AMD. In addition, it includes 4.6 TOPs of its own (for FP32, 4x that amount for FP8)—it is part of what the company says is a total system TOPS of 75 when leveraging Qualcomm’s AI Engine software.
On top of all these core specs, new systems with Snapdragon X Elite will have the ability to include 5G support—both for sub-6 and mmWave bands—if vendors choose to include a
Qualcomm X65 modem that can connect via an M.2 card or be built onto the PC’s motherboard. PCs can also optionally integrate the company’s FastConnect 7800 WiFi chip to offer full WiFi 7 support. I wish these connectivity technologies were standard instead of optional, as I’ve argued many times that strong connectivity is often more important than pure compute capabilities for modern work and consumer applications. Still, good to see Qualcomm making it easy to include if their PC OEM partners choose to do so.
Finally, Qualcomm is rounding things out with an always-on Sensing Hub that integrates its own micro-NPU. This enables applications like presence detection, image processing on the webcam, and more. In addition, the Sensing Hub includes support for the company’s Snapdragon Sound audio technology.
Collectively, there’s little doubt that Snapdragon X Elite offers an impressive array of AI-focused capabilities and definitely adds to the momentum around AI PCs. As with all the previous generation Qualcomm-powered PC efforts, there will likely be lingering questions about software compatibility as these systems start to come to market. Thankfully, progress has been made here, with Microsoft continuing its work to ensure that all of its key offerings run natively. Qualcomm has also been working with developers like Adobe and others to make their application run efficiently on Arm architectures.
While it won’t be an easy one, it looks like the crown for the PC semiconductor industry is becoming a multi-horse race.
Here's a link to the original column: https://www.linkedin.com/pulse/qualcomms-snapdragon-x-elite-solidifies-new-era-ai-pcs-bob-o-donnell-zr0df/ Bob O’Donnell is the president and chief analyst of TECHnalysis Research, LLC a market research firm that provides strategic consulting and market research services to the technology industry and professional financial community. You can follow him on LinkedIn at Bob O’Donnell or on Twitter @bobodtech. |
